Head-counter device and method for processing digital images

ABSTRACT

A head-counter device (100) comprising a digital camera (1) adapted to provide a first digital image (IM1) representative of a counting zone (STR) of persons, the first image defining a first horizontal dimension (N) and a first vertical dimension (M), and a cropping module (7) configured for: analyzing the first image (IM1) and identifying a noise area (PCR) according to at least one of the following features: pixel light intensity, pixel color and/or presence of predefined patterns, cropping the noise area (PCR) from the first image (IM1) to obtain a second image (IM2) without the noise area, the noise area (PCR) being a peripheral portion of the first image having said first horizontal dimension and having a second vertical dimension (M−DSK+SM) shorter than the first vertical dimension.

TECHNICAL FIELD

The present invention relates to digital image processing techniquesthat can be used for devices known as head-counters.

STATE OF ART

The counting of persons is interesting, among other things, forstatistical analyses aimed at commercial evaluations. Indeed, personsstationed in front of a shop window or walking through a shopping centercan be counted in order to assess the attractiveness of such places.

Known head-counter devices automatically count persons who stop ortransit in a specific area. The count is carried out by processingdigital images that show the area of interest.

US-A-2010232644 describes a method for analyzing the number of personsobserving a billboard based on the processing of images obtained from adigital camera.

U.S. Pat. No. 5,465,115 describes a method for monitoring of personsthat simultaneously enter and exit from a pedestrian area such as theentry of a commerce location, based on the processing of video images.

U.S. Pat. No. 9,449,506 B1 describes a head-counter device whichautomatically detects an area of interest by means of an object movementanalysis which analyzes the movement of persons moving through the areaof interest.

US 2016/0110613 describes a head counter device which counts personspresent in a preset region of interest and U.S. Pat. No. 6,282,317describes a method for recognizing skies, patterns, vegetations andpersons in images.

However, said known devices and methods are relatively complex and/orhave structural precision and/or efficiency limits in counting thenumber of persons, in particular in a crowded and limited area, comparedto the whole area framed by the camera.

SUMMARY OF THE INVENTION

The present invention addresses the problem of providing a technique forcounting persons, usable in open space (i.e. outdoor) applications thatcombines counting precision and computational speed.

In particular, the present invention relates to a head-counter device asdefined by claim 1 and by its preferred embodiments, defined by thedependent claims.

The subject of the present invention is also a method for processingdigital images as defined by claim 13.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is hereinafter described in detail, by way of anon-limiting example, with reference to the accompanying drawings, inwhich:

FIG. 1 shows schematically an example of a head-counter device;

FIG. 2 refers to some example steps of the digital image processingmethod in accordance with the present invention.

DETAILED DESCRIPTION

While the invention it is susceptible of various modifications andalternative constructions, some particular embodiments are shown in thedrawings and will be described in detail below. In the presentdescription, similar or identical elements or components will beindicated in the figures with the same identification symbol.

FIG. 1 shows a head-counter device 100 comprising a digital camera 1 anda digital image processing module 2. The digital camera 1 is,advantageously, a high resolution camera capable of capturing images,for example, up to a distance of 60 meters.

The processing module 2 is, for example, a microcontroller suitable forrunning a digital image processing software.

The head-counter device 100 comprises, preferably, an electric battery 5or can be connected to an external electric power source, for itselectric supply. Moreover, the head-counter device 100 can be providedwith a communication module 6, suitable for transmittingdata/information outside the device itself and, in particular, to acloud system. In particular, this data/information is “metadata” andthus does not include digital images provided by the digital camera 1but includes only the aggregate result that comes from the digital imageprocessing which will be described by way of example in the following.

The head-counter device 100 may have a container 3 which houses theabove-mentioned components and is also provided with mechanical meansfor its fastening to a support.

The head-counter device 100 is suitable for counting the presence ofpersons in an outdoor area and is therefore suitable to operate in theoutdoor area (even if in a non-exclusive way). The head-counter device100 can be used for counting persons crossing or walking through adetermined area such as, for example: a square, a road, a sidewalk.Furthermore, the head-counter device 100 can be used not only forcounting persons but also for counting vehicles or animals. The term“persons” means, for the purposes of the present invention, humans,vehicles or animals.

The head-counter device 100 can be mounted outside any static object(such as for example a pole, a billboard and the facade of a building),on a moving vehicle (such as a car, a bicycle, a bus) or it can be usedas an item that can be carried by a person.

The processing module 2 is configured to receive a digital imageprovided by the camera 1, to recognize the presence of the personspresent in the image and to count their number. In greater detail, theprocessing module 2 is provided with a cutting or cropping module 7(CROP), configured for the removal of noise, and with an analysis andcount module 8. According to an embodiment, the analysis and countmodule 8 includes a person identification module 9 (P-ID), a person faceidentification module 10 (F-ID) and a person profiling module 11 (P-PR).

The modules 7-11 listed above can correspond to code portions of adigital image processing software which can also be marketedindependently from the camera 1 and which can be run, for example, in aprocessor far from the camera itself.

The operation of the head-counter device 100 will be described belowalso with reference to FIG. 2 which shows, by way of example, theprocessing of a digital image. The functions performed by modules 7-11correspond to steps of a digital image processing method.

In this example, it is assumed that the head-counter device 100 ismounted on a wall of a building and that it frames an external scenerelating to a street, with side buildings and sidewalks on which personscan walk.

The digital camera 1 provides an image according to a predeterminedfrequency. It is assumed that the digital camera 1 supplies to theprocessing module 2 a first image IM1 having one first horizontaldimension N (along a horizontal axis x) and a first vertical dimension M(along a vertical axis y). The N and M values can be the same ordifferent. The N×M dimensions of the first image IM1 refer, as it isevident to the skilled person in the art, to the number of pixelscontained in the first image IM1.

The cropping module 7 receives the first image IM1 (namely the digitaldata defining it) and analyzes it in order to identify a noise area PCR.The noise area PCR is an area of the image that inherently is of nointerest for the counting of persons and is associated with scenariosincompatible with the presence of persons transiting.

Incompatible scenarios are those areas where it is not presumable toidentify persons transiting. For example, noise areas are those portionsof the images relating to the sky, vegetation (for example trees) andwalls of buildings. The noise area may also relate to a combination ofsky areas, building walls or tree/vegetation extremes.

The noise area PCR is identified according to a processing that takesinto account at least one of the following features: light intensity ofthe pixels, pixel color, presence of predefined patterns.

According to a particular example, the cropping module 7 identifies inthe first image IM1 areas related to the presence of sky SK (namely ofan area of the image related to a no-interest area for the counting ofpersons). The identification of the sky SK is carried out by analyzingthe light intensity of the three RGB (Red Green Blue) channelsassociated with the pixels, assuming that portions of images adjacent toone another which are characterized by pixels with a predominance of theblue color are relative to the sky.

Advantageously, in order to avoid false detections for example due toblue objects (cars) or areas with reflections of the sky, the sky SK isidentified, among other areas of blue color which are present, as theone which has the largest area and/or has a higher density of bluepixels.

If a significant presence of the sky is determined in the first imageIM1, a peripheral portion PCR which contains this sky, extends over theentire horizontal dimension N and has a vertical length equal to a valueDSK determined according the extension of the sky, is removed from thisimage. For example, the portion PCR indicated in FIG. 2 by a hatch,having horizontal coordinates x comprised between 0 and N and verticalcoordinates y comprised between M and M−DSK, can be cropped.

Advantageously, once the portion PCR of the sky to be cropped isidentified (and therefore its value on the y axis), a safety margin SMis subtracted therefrom in order to reduce the portion PCR of the sky tobe cropped: PCR-SM.

If the portion of the image associated with the sky does not cover asignificant area with respect to the dimension M (for example, itextends along the vertical axis y for a length significantly shorterthan M) an analysis is also provided for identifying the presence ofareas that show facades of buildings BDG at heights (along the y axis)higher than those that are of interest for the purpose of countingpersons. For example, if the detected portion of the sky, in terms ofvertical dimension on the vertical axis y, is less than 20% of theentire vertical dimension M of the image, then the analysis on thefacades of the buildings is applied.

In this regard, an exemplifying situation is considered, in which thecamera 1 is mounted on a pole/building at a height higher than that ofthe man, in order to be able to frame the area of interest this cameramust be tilted downwards at a certain angle. Since the image istherefore sampled at a downward angle, the portion of the sky should bereduced or even be absent. In this exemplifying situation, the facadesof the buildings (if present in the image and therefore correctlydetected) are used for the subsequent processing steps.

The walls of buildings can be identified through the recognition ofpredefined patterns, relative to typical architectural elements of thebuildings, such as, preferably, the windows. In fact, the windows arerecognizable by the presence of horizontal lines or contours(windowsills or architraves) and vertical (the jambs).

The typical window structure also allows to identify a horizontal lineconnecting the sills of more windows of a same housing level (forexample, the first floor or the second floor). This horizontal line mayconstitute a lower limit for the definition of the noise area PCR. Forexample, this line may correspond to a threshold value TRH (expressed asa number of pixels).

Once the noise area PCR, including in the example of FIG. 2 both aportion of sky SK and a portion relative to the facades of the buildingBDG, is identified, the noise area PCR is cut out, namely removed fromthe first image IM1.

To locate a noise area PCR with the presence of vegetation (for example,a background with trees) the processing is based on the light intensityof the pixels of the first image IM1, associated with the green color.Thus, also the noise area PCR related to vegetation (occupying aperipheral portion of the first image IM1 at a height along the verticaly-axis higher than the threshold value TRH) is cropped as in the case ofthe sky SK or of the facades of buildings BDG.

A second image IM2 having a reduced number of pixels compared to thefirst image IM1, namely not with a size N×M but with a reduced size:(M−DSK+SM))×N, is obtained from the cropping.

It should be noted that, advantageously, also a resizing step can beforeseen (for example carried out by the same cropping module 7), inwhich the second image IM2 resulting from the cutting or cropping phaseis returned to the size of the first image IM1. This resizing can bedone by adding to the second image IM2 additional pixels generatedstarting from the pixels already existing in this image resulting fromthe cropping. These additional pixels can be obtained, for example, froma linear interpolation of adjacent pixels or from averages on adjacentpixels.

It is remarked that the set of the cropping step carried out by thecropping module 7 and the subsequent resizing to the initial value N×Mcorresponds to a digital zoom operation carried out on the portion ofthe image relevant for the purposes of counting persons and without (orsubstantially lacking of) sky, vegetation and walls of buildings.

Crop and resize operations can be performed by means of a softwaresuitably configured for the functions described above and based, forexample, on image processing techniques known to the expert in thefield.

Following to the cropping and resizing (7) a third digital image IM3 isobtained which is supplied to the analysis and count module 8 and thento the person identification module 9.

The person identification module 9 processes the third image IM3 byapplying an algorithm (which can be of a known type), suitable forrecognizing the presence of persons and allows to identify a certainnumber of persons PERS1-PERS3 in this image For example, a number ofpersons P equal to 3 is identified. It is also possible that an imagedoes not portray any person and the person identification module 9returns a counting value P equal to 0.

The person face identification module 10 processes the portions ofimages relating to persons PERS1-PERS3 identified above so as todistinguish their faces: for example the two faces V1 and V2.

The person profiling module 11 processes (by applying an algorithm thatmay be of known type) images relating to the faces V1 and V2 andgenerates profiling information, relating to the persons framed by thecamera 1. In particular, this profiling information relates to gender(MEN or WOMEN) of the framed person or his/her estimated age (AGE), forexample, expressed by a range.

The person profiling module 11 can also be equipped with an algorithm(for example, of a known type) which allows to estimate the attention ofthe analyzed person for a reference item such as, for example, a shopwindow, a billboard, a display or other. This estimate can be made, forexample, on the basis of the orientation of the head and the eyes andthe time spent by the person in observing the item.

Both the count number P and the profiling data (metadata) related togender and age and/or to the attention can be stored locally or,preferably, sent to a cloud server (not shown) through the communicationmodule 6.

The processing carried out by the analysis and count module 8 can alsobe implemented by means of a neural network. For example, an R-FCNnetwork (Region-based Fully Convolutional Network) can be used, such as,in particular, a network like the one described in the document: JifengDai et al., “R-FCN: Object Detection via Region-based FullyConvolutional Networks”, —21 Jun. 2016.

It should be noted that, thanks to the cropping operation as describedabove, the processing performed by the analysis and count module 8 isfully suitable for outdoor applications, with high performance. In fact,the elimination of the noise areas PCR performed as described aboveallows the identification and counting steps (9), as well as theprofiling steps (11) to be performed on a digital image (i.e. the thirdimage IM3) in which the portion with possible presence of persons isincreased (in relative dimensions), with respect to the relativedimensions that the same portion has within the first image IM1. Thismakes the processing carried out by the analysis and count module 8 moreprecise, less computationally burdensome and faster than those thatwould have been obtained by directly processing the first image IM1.

It is also worth observing that since the recognition and cropping ofthe noise areas PCR is performed automatically by the software of thecropping module 7, complex configuration operations of the head-counterdevice 100, whose purpose is to maximize the framing of the areaaffected by the passage of persons with respect to the sky or otherareas of no interest, are not required during installation of the samein a given position.

Furthermore, the automatic cropping of the sky, the facades of buildingsand the trees, carried out by the head-counter device 100, allows itsuse also in motion (on vehicles or carried by persons).

It should be noted that the possibility of processing the imageslocally, namely within the head-counter device 100, appears advantageousin that it eliminates the delays due to the transmission of the datacorresponding to the digital images taken, typical of a remoteprocessing.

It should be observed also that the local (namely at the head-counterdevice 100) image processing allows not to violate privacy norms byavoiding that any person may access to the processed images, thus makingit impossible to identify the framed persons. Once processed, the imagecan be deleted and only the metadata corresponding to the number ofpersons detected, gender and age are transmitted in an aggregate mannerto the cloud server.

Possible variations or additions can be made by those skilled in the artto the embodiment described and illustrated herein while remainingwithin the scope of the following claims. In particular, furtherembodiments may comprise the technical characteristics of one of thefollowing claims with the addition of one or more technical featuresdescribed in the text or illustrated in the drawings, taken individuallyor in any combination thereof.

The invention claimed is:
 1. A digital image processing method,comprising carrying out in order the following steps: receiving a firstdigital image (IM1) representing a counting zone (STR) of persons, thefirst image defining a first horizontal dimension (N) and a firstvertical dimension (M); carrying out a pre-processing phase comprisingthe following steps: analyzing the first image (IM1) without searchingin order to recognize persons, but only in order to identify a noisearea (PCR), wherein said noise area is an area of the image that isassociated with scenarios incompatible with the presence of persons,according to the following features of said noise area: pixel lightintensity, pixel color, presence of predefined patterns, identifyingsaid noise area (PCR) according to the following backgrounds: sky,vegetation, buildings, shown in the first image (IM1) by means of atleast one of the following steps: identifying the sky background (SK)according to the light intensity of the pixels of the first image (IM1)associated with the blue color, identifying the vegetation backgroundaccording to the light intensity of the pixels of the first image (IM1)associated with the green color, identifying the building background(BDG) according to patterns of architectural elements recognizable insaid first image (IM1); cropping the noise area (PCR) from the firstimage (IM1) to obtain a second image (IM2) without the noise area, thenoise area being a peripheral portion of the first image having saidfirst horizontal dimension and having a second vertical dimension(M−DSK+SM) shorter than the first vertical dimension; processing saidsecond imagine (IM2) recognizing at least one person shown in the secondimage (IM2) and providing a count number (P) of said at least one personin said counting zone (STR).
 2. The digital processing method of claim1, comprising the step of resizing the second image (IM2) without thenoise area (PCR) to obtain a third image (IM3) having the firsthorizontal dimension (N) and the first vertical dimension (M); whereinsaid step of processing is carried out on said rectangular third digitalimage (IM3) for recognizing at least one person shown in the thirddigital image (IM3) and for providing a count number (P) of said atleast one person in said counting zone (STR).
 3. The digital processingmethod of claim 2, comprising the step of obtaining the third image(IM3) according to the generation of additional pixels calculated byprocessing further pixels present in the second image (IM2).
 4. Thedigital processing method of claim 3, comprising the step of generatingthe additional pixels by performing a linear interpolation of saidfurther pixels present in the second image (IM2).
 5. The digitalprocessing method of claim 4, comprising the steps of: identifying theface of a person represented in the third image (IM3); estimating ageand/or gender of a person represented in the third image (IM3);providing an attention estimate value of a person represented in thethird image (IM3) towards a predefined item.
 6. The digital processingmethod of claim 1, further comprising the step of identifying the sky(SK) or the vegetation is identified as the area that has the widestarea and/or has a higher pixel density compared to other areas of blueor green color, respectively, present in the first image (IM1).
 7. Thedigital processing method of claim 1, further comprising the followingstep: if a significant presence of the sky (SK) or vegetation in thefirst image (IM1) is determined, removing from the first image (IM1) anoise area (PCR) containing such sky (SK) or vegetation, which extendsover the entire horizontal dimension (N) of the first image and whichhas a vertical length equal to a value (DSK) established according theextension of the sky (SK) or of the vegetation.
 8. The digitalprocessing method of claim 1, comprising the step of identifying thebuilding background (BDG) by a horizontal line (TRH) which forms a lowerlimit for the definition of the noise area (PCR).
 9. The digitalprocessing method of claim 1, comprising the step of subtracting asafety margin (SM) from the noise area (PCR) to reduce the rectangularsecond image (IM2) to be cropped from the first image (IM1).